Click any tag below to further narrow down your results
Links
Dylan Patel and Dwarkesh Patel discuss how OpenAI and Anthropic are on track to command most of the world's usable computing capacity within a few years by outbidding everyone else, thanks to their ability to monetize inference at much higher margins than the raw cost of compute. They also explore whether the $10+ trillion in AI infrastructure spending by decade's end could trigger a sovereign debt crisis.
- OpenAI and Anthropic are capturing 40-50% of new compute capacity next year (up from 30% this year), and at current growth rates will control most of the world's usable computing power by end of 2028, since they're deploying the most efficient latest-generation chips while competitors use older hardware.
- These labs have flipped from venture-funded losses to profitability by achieving 50x revenue per megawatt of compute (Anthropic), allowing them to reinvest all profits into training and continuously outbid other companies for scarce compute capacity.
- The concentration of compute in two companies raises questions about whether massive hyperscaler debt could drive up interest rates globally, push non-AI countries into bankruptcy, and whether any force can counteract the economics pushing toward centralization.
The author argues that despite improvements in open-weight models, most AI inference will remain in datacenters because local models can't match frontier performance and are actually more expensive to run. Batching hundreds of users' requests together and specialized datacenter GPUs make cloud inference roughly 30x more efficient than running models at home, and users will always prefer the strongest available model in their budget.
- Datacenter inference beats local by ~30x on efficiency due to request batching and specialized GPUs (e.g., B200 vs RTX 4090)
- A home GPU rig's upfront cost plus $50-300/month in power outweighs just paying for years of API access
- Users always gravitate to the strongest model they can afford, so smaller local models keep losing ground even as they improve
- Local models will persist only in niches like low-latency voice interfaces, privacy-focused use, or unreliable internet—not as the dominant paradigm
The article argues that enterprises should measure AI infrastructure economics by cost per token rather than raw compute metrics like FLOPS per dollar. It shows how maximizing delivered tokens—through hardware, software and system optimizations—drives down real-world cost and boosts revenue, citing NVIDIA Blackwell’s 35× lower token cost versus Hopper.
- Cost per token (total infra cost ÷ tokens generated), not FLOPS/dollar or GPU hourly rate, is the real measure of AI infrastructure efficiency.
- Blackwell GB300 NVL72 costs almost 2x more per GPU-hour than Hopper H200 ($2.65 vs $1.41), but delivers 65x the tokens/sec per GPU (6,000 vs 90).
- That throughput gap translates to 50x more tokens per megawatt and a 35x lower cost per million tokens ($0.12 vs $4.20).
- Techniques like FP4 precision, speculative decoding, KV-cache offloading, and disaggregated serving are necessary, not optional, to actually achieve these lower token costs.